: :ODMA\MHODMA\iManage;282823 ; 1 

NSP/LRR/JPH/nzk 

01/16/02 



PATENT APPLICATION 
Docket No.: 2328.2004-001 



-1- 



Date: 




Express Mail Label No._ 



Inventors: 



Samson Mil'shtein and Joel Therrien 



Attorney's Docket No.: 



2328.2004-001 



METHOD OF CORRELATION OF 



IMAGES IN BIOMETRIC APPLICATIONS 



Q 




RELATED APPLICATION 

This application claims the benefit of U.S. Provisional Application No. 
60/262,093, filed on January 16, 2001, entitled "Method of Reliable Correlation of 
Images in Biometric Applications," by Mil'shtein et al. The entire teachings of the 
above application are incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

Image correlation is useful in several areas of practical application. For 
example, image recognition systems often use correlation of an acquired actual image 
and a reference image for comparison. In particular, images of biometric objects such 
as faces, retinas scans, fingerprints, and hand or palm prints, are compared using the 
image recognition systems to identify an individual or verify whether an individual is 
the person he claims to be for security. Image correlation is typically performed using 
Fast Fourier Transforms (FFTs), image shifting, or optical transformation techniques. 

SUMMARY OF THE INVENTION 

These techniques for image correlation, however, require extensive processing 
of the images in hardware or software. The computational load increases significantly 
as the image resolution increases. For an image having N x N pixels, for example, FFT 
techniques require on the order of N 2 logN iterations while image shifting techniques 
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require A 2 N 2 iterations, where A is the length of the correlation search in pixels. 
Further, image correlation requires the actual image to be in alignment with the 
reference image so that the coordinate system of the actual and reference images are in 
the same orientation and position. Furthermore, the presence of background noise in the 

5 images makes the pixel-by-pixel comparison of images very difficult. 

The image correlation process and apparatus of the present invention provides 
several orders of magnitude in increased processing speed over known image 
correlation processing techniques by compressing images into image projections and 
employing deviation of symmetry of image projections. In addition, the image 

10 correlation system of the present invention does not require massive computational 
power nor a powerful processor. Instead, much smaller chips like memory shifters or 
Digital Signal Processing (DSP) chips can be used. In addition, the present invention 
reduces a significant amount of noise usually found in an optical image for a more 
accurate comparison. Further, a precise alignment of images being compared to avoid 

1 5 errors in prior image correlation methods is not needed in the present invention. 

The present image correlation process may be used to compare images. The 
image correlation process includes compressing a first image into a first image 
projection and a second image into a second image projection. The first and second 
image projections may be in one-dimensional format to simplify the application of 

20 correlation functions. The images may be compressed by summing the pixels in each 
column and row for x and y projections respectively. 

Once the images are compressed, the deviation of symmetry of the first and 
second projections are determined to compare the images. The deviation of symmetry is 
determined by providing a correlation function and locating a correlation peak. The 

25 correlation function is performed at a first point near the correlation peak and a second 
point near the correlation peak to determine first and second correlation results. In 
addition, the correlation function is performed at the correlation peak to determine the 
peak correlation result. The first and second points maybe on the opposite sides of the 
peak and at about the same distance from the peak. The deviation of symmetry is 
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determined by computing a difference between the first and second correlation results 
and dividing the difference by the peak correlation result. 

A predetermined threshold is provided to determine whether the first and second 
images are the images of a same object. A few percent deviation from symmetry may 
5 indicate that the correlation function has compared the images of two different objects. 
However, if the deviation of symmetry is small and less than the threshold value, then 
the first and second images are the images of the same object. 

The image correlation process may be used to authenticate an acquired image by 
comparing it with a predetermined reference image for security in sensitive government 
10 and commercial installations. In addition, matching for personal identification has 

application wherever a person's identity needs to be identified or verified, such as in the 
control of access to physical locations, such as airports, industrial locations and in the 
home. 

The correlation function includes a displacement parameter to align the first and 
15 second images. In addition, background noise in image projections are reduced by 
subtracting an average background constant from the correlation function. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other objects, features and advantages of the invention will be 
apparent from the following more particular description of preferred embodiments of 
20 the invention, as illustrated in the accompanying drawings in which like reference 

characters refer to the same parts throughout the different views. The drawings are not 
necessarily to scale, emphasis instead being placed upon illustrating the principles of the 
invention. 

FIG. 1 is a flow chart of an image correlating process according to the invention; 
25 FIG. 2 is a flow chart of a deviation of symmetry process of FIG. 1 ; 

FIGs. 3 A - 3B are diagrams showing the deviation of symmetry of x and y 
projections of a correlation function when the images of the same object are compared; 
and 
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FIGs. 4A - 4B are diagrams showing the deviation of symmetry of x and y 
projections of a correlation function when the images of two different objects are 
compared. 

DETAILED DESCRIPTION OF THE INVENTION 
5 A description of preferred embodiments of the invention follows. 

The present invention relates to correlating images of biometric objects. The 
present invention uses the deviation of symmetry of correlation functions to compare 
images. In general, a correlation of images yield a result, in the form of a discretized 
function of the offset of the data sets being compared. If the images being compared are 

1 0 describing the same object then there is a symmetry of correlation functions. The 
degree of symmetry provides the closeness of fit between the images. For example, a 
few percent deviation from symmetry indicates that the correlation function is 
comparing two different images. 

In particular, the present image correlation process maybe used to compare 

15 prerecorded reference image and an actual acquired image of a biometric object such as 
fingerprints. The reference image may be stored in memory and the actual image may 
be acquired using an image capturing device. 

FIG. 1 is a flow chart of an image correlating process 2 according to the 
invention. The process begins at step 10. Since it is computationally intensive to apply 

20 recognition algorithms to images, resulting from two dimensional nature of images, the 
images are compressed into one-dimensional format projections at 12 and 14. The 
projections preserve sufficient details of the images to make the image recognizable. 
The compression may consist of taking the projection of an image in the x and y 
directions. For example, the x projection of the image is computed by summing the 

25 pixels in each column. Similarly, the y projection of the image is computed by 

summing the pixels in each row. At 12, the first image is compressed into a first image 
projection. Similarly, at 14, the second image is compressed into a second image 



projection. The first and second image projections may each include x and y projections 
of the respective images. 

In the case of fingerprints, this compression process may obliterate information 
about the ridges of the fingerprint, but does not obliterate information about breaks, 
bifurcations and curvature of the fingerprint. The lack of information about the ridges 
does not pose a problem because many recognition algorithms focus exclusively on 
breaks and bifurcations. The breaks and bifurcations appear in the form of short period 
changes in the projections. The curvature information appear as longer period 
variations in the projection. This projection technique may also be applied to the 
Fourier transform of the images with no modification. 

Once the images have been compressed, the deviation of symmetry of the first 
and second projections is determined at 16. FIG. 2 is a flow chart of a deviation of 
symmetry process 16 of FIG. 1. The deviation of symmetry is determined using a 
correlation function. 

A correlation function produces a linear and discretized set of data, which is a 
function of the relative displacement between the two projections: 

max 

C(J) = ]jr projection (n) x projection 2 (n + S) 

[1] 

where n represents the individual components of a projection and 5 is the displacement 
of the image. 

According to Equation 1, in the event that the projections are the same, e.g. the 
images are the same, then the correlation function C(6) is symmetric around 5=0. If one 
of the images is shifted, the function is symmetric around 6=-d where d is the 
displacement along the projection of the second image relative to the first. 

One of the problems in correlating images, especially for fingerprints, is caused 
by images that have a uniform background. As 5 increases, the overlap of the two 
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projections decreases. Each projection can be considered as a function riding on top of a 
uniform background, represented as a constant average. Then, the correlation function 
can be rewritten as such: 



max = | datalength - 5 | projection(n) = prj(n) + avg 



5 C(S) = \ £ prj\ (n) x prj 2 (n) + prj x (n) x avg 2 + prj 2 (n) x avg { } + maxx avg x x avg 2 
U=o 

[2] 

As shown in Equation 2, the correlation function has a component that linearly 
decreases as a function of 5. If the background constants avgj and avg 2 are larger than 
the variance of prj^n) and prj 2 (n), then the component of the correlation function due 

10 to the background overwhelms the component due to the fingerprint data. To overcome 
this, the average value may be subtracted out. Thus, the background value is subtracted 
from the projection pixels. 

The sensitivity of the correlation function for short range variations in the 
projections extend near a correlation peak. For example, the part of the correlation 

1 5 function which is sensitive to the breaks and bifurcations in the fingerprint extends only 
a short distance from the peak. However, the sensitivity of the correlation function for 
longer-range variations in the projections extends much farther. For example, the 
correlation function is sensitive to the general curvature of the fingerprint across the 
whole function. Thus, if the images of the same object is compared, then there is a 

20 symmetry near the peak. However, if the images of different objects are compared, then 
there is a deviation from symmetry due to the sensitivity of the correlation function near 
the correlation peak. 

The symmetry of image projections is determined by locating the peak value of 
the correlation function C(6) and folding the correlation function. As illustrated in 

25 Equation 3 below, the correlation function is folded by performing the correlation 



2328.2004-001 



-7- 



function at A + 6 and A - 8, and calculating the difference between the two, where A 
represents the correlation peak. The correlation peak is associated with A=-d. Thus, a 
symmetry function is constructed: 



|C(A + <5)-C(A-*)l 



[3] 



The symmetry function is normalized by dividing by the peak correlation result 
determined by performing the correlation function at C(A). With the normalization, the 
symmetry function returns the percentage deviation of symmetry as a function of 
distance from the center. In practicality, the two images are not exactly the same. 

1 0 Therefore, the center of symmetry may not be located exactly at the peak in the 
correlation function. 

Similarly, to determine the deviation of symmetry of the first and second 
projections, a correlation peak is determined at 22. At 24, the correlation function is 
computed at a first point near the correlation peak, A + 6, to determine a first correlation 

1 5 result. Similarly, at 26, a second correlation result is determined for a second point near 
the correlation peak, A - 8. In addition, a peak correlation result is determined by 
performing the correlation function at the correlation peak, A, at 28. Once the first and 
second results are determined, the difference between the first and second results is 
computed, at 30, to determine the deviation of symmetry. At 32, the difference is 

20 divided by the peak correlation result to provide the percentage deviation of symmetry. 
The symmetry process ends at 34. 

Referring again to FIG. 1, once the deviation of symmetry is determined at 16, 
the deviation of symmetry is compared to a threshold predetermined by a user to 
determine whether the first and second images are the images of a same object. 

25 FIGs. 3 A and 3B illustrate the deviation of symmetry of x and y projections of a 

correlation function when the images of the same object are compared. In particular, 



FIG. 3 A illustrates a plot of the deviation of symmetry (in %) versus the distance from 
the center of symmetry for the x projection of the correlation function. FIG. 3B 
illustrates a plot of the deviation of symmetry (in %) versus the distance from the center 
of symmetry for the y projection of the correlation function. FIGs. 3 A and 3B show a 
small degree of deviation from symmetry (10°). In fact, almost no deviation is observed 
in the y projection of FIG. 3B. Accordingly, the compared images are the images of the 
same object. The correlation function maybe an autocorrelation function to compute 
the degree to which the images replicate each other. In addition, one of the images was 
rotated about 5°. 

FIGs. 4A - 4B are diagrams showing the deviation of symmetry of x and y 
projections of a correlation function when the images of two different objects are 
compared. In particular, FIG. 4A illustrates a plot of the deviation of symmetry (in %) 
versus the distance from the center of symmetry for the x projection of a correlation 
function. FIG. 4B illustrate a plot of the deviation of symmetry (in %) versus the 
distance from the center of symmetry for the y projection of the correlation function. A 
high degree of deviation from symmetry is observed (10 1 ). Accordingly, the compared 
images are the images of two different objects. The correlation function may be a cross 
correlation function to compute the degree to which the images differ. The images were 
not rotated in this case. 

While this invention has been particularly shown and described with references 
to preferred embodiments thereof, it will be understood by those skilled in the art that 
various changes in form and details may be made therein without departing from the 
scope of the invention encompassed by the appended claims. 



